Add race condition fix in critical section logic#1248
Merged
andystaples merged 1 commit intoOct 9, 2025
Conversation
- Fix making a specific entity locking race condition error correctly
sebastianburckhardt
approved these changes
Oct 7, 2025
|
Am I the only one who feels like a test is missing ? 🙃 |
Contributor
Author
We will want to add testing for this change, but since DurableTask.Core does not expose entity functionality (and even if it did, I don't think the orchestration APIs don't even allow the same kind of race condition here) we will add the tests in the E2E test suite in azure-functions-durable-extension. |
cgillum
approved these changes
Oct 9, 2025
cgillum
left a comment
Member
There was a problem hiding this comment.
I’m good with this change based on manual testing. Integration testing will need to be done separately in another repo.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR addresses a very specific bug with a race condition caused by improper handling of orchestration flow.
The bug exists with this kind of setup in Durable Functions using the dotnet-isolated programming model, and having more than one input to tags:
This is non-supported behavior, as awaiting WhenAll on tasks that are comprised of multiple Durable operations is likely to cause non-determinism exceptions, and also has issues with entity locks in particular as one orchestration should only hold one entity lock at a time.
However, this pattern today throws a NullReferenceException in OrchestrationEntityContext.RecoverLockAfterCall as RecoverLockAfterCall gets called while the lock is still pending due to the race condition, so availableLocks has not been initialized. The fix in this PR causes the orchestrator to instead throw
System.InvalidOperationException: Must not enter another critical section from within a critical section.which is more helpful in identifying the race condition in the tasks.